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ABSTRACT 



To obtain practitioner input on the research agenda for 
adult basic education (ABE) , practitioner leaders from the Practitioner 
Dissemination and Research Network ( PRDN) conducted nine focus groups with 
ABE practitioners in their states. The focus groups examined three topics: 
issues that concern practitioners in ABE; ways practitioners see their 
concerns being addressed by current and future research; and the PRDN's role. 
Overall, the practitioners in ABE were looking for answers and stepping 
stones to improve ABE practice. They expected research to address their 
concerns and were generally anxious to work with researchers in designing and 
conducting studies that address their concerns. The following were among the 
practitioner concerns identified: student recruitment, retention, and 
motivation; program and policy (program structure and funding) ; curriculum 
and instruction; assessment of student and program performance; "special" 
students' needs; staff issues and professional development; impact of ABE on 
students' lives outside the classroom; and ways of working with a variety of 
learners in one class. (Appended are the following: focus group protocol; 
tables detailing the focus group participants by state and number of affinity 
diagram items by state; questions for the research agenda; and categories of 
issues of concern to focus group participants.) (MN) 
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Executive Summary 

The National Center for the Study of Adult Learning and Literacy (NCSALL) is 
committed to conducting and disseminating research that is used to improve the 
practice of adult basic literacy education. As part of this effort, NCSALL has 
developed the Practitioner Dissemination and Research Network (PDRN). This 
network of practitioner leaders builds research connections between practitioners 
and NCSALL researchers. In the summer of 1997, practitioner leaders conducted 
focus groups with practitioners in their states. This report summarizes the results 
of these nine focus groups. 

These focus groups were part of NCS ALL’s work in developing a comprehensive 
research agenda for the field of adult literacy, one of two research agendas 
NCSALL is helping develop for the field of adult basic education. In these 
agendas, the input of practitioners is combined with the input from administrators, 
policymakers and researchers. This document, however, provides an opportunity 
to hear directly from practitioners and to learn about the realities of practitioners’ 
work 

The PDRN focus groups gathered data on three related questions: 

• What issues concern practitioners in adult basic education? 

• How do practitioners see these concerns being addressed by current and future 
research? 

• What do practitioners see as the role of the PDRN in their states? 

The concerns of the teachers and program administrators fall into eight categories: 

• Issues of student participation: recruitment, retention, and motivation 

• Program and policy issues, particularly program structure and funding 

• Curriculum and instruction issues 

• Assessment of students and measurement of program performance 

• The needs of "special” students, including those with learning difficulties, 
teens, and the elderly 

• Staff issues and professional development 

• The impact of adult basic education on students’ lives outside the classroom 

• Working with a variety of learners in one class 

Participants in these groups asked for research that: 

• Addressed their areas of concern 

• Has implications for practice 
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• Is conducted in a variety of sites in both rural and urban settings 

• Includes practitioners in the research 

• Is disseminated in a variety of ways 

• Is reported concisely in "user-friendly” language 

Practitioners in adult basic education are looking for answers and for stepping 
stones to improve practice. They expect research to address their concerns and 
many are anxious to work with researchers in designing and carrying out studies 
that do so. 
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I. Introduction 

In the summer of 1997, the National Center for the Study of Adult Learning and 
Literacy (NCSALL), through its Practitioner Dissemination and Research 
Network (PDRN) initiative, organized focus groups in nine states to collect 
information about adult basic education. These focus groups were part of 
NCSALL’ s effort to help the field of adult basic education articulate a 
comprehensive research agenda. This report summarizes the findings of these 
focus groups and offers conclusions and recommendations. The results from 
these nine focus groups provide a rich insight into the issues faced by adult basic 
education (ABE) practitioners and into their hopes for NCSALL and other 
researchers in the field of adult basic literacy. 

NCSALL is a collaboration between Harvard Graduate School of Education, 
World Education, the Center for Literacy Studies at the University of Tennessee, 
Rutgers University and Portland State University. The goal of NCSALL is to 
help adult basic education practitioners better serve adult learners by pursuing 
basic and applied research in the field of adult basic education; building 
partnerships between researchers and practitioners; disseminating research and 
best practices to practitioners, scholars, and policymakers; and working with the 
field to develop a comprehensive research agenda. 

NCSALL is a partner in two collaborations that are developing research agendas 
for the field. One collaboration involving NIFL and OVAE/DAEL is developing 
an agenda for the broad field of ABE, English as a second language (ESOL) and 
adult secondary education. In a second collaboration, NCSALL is working with 
NCLE on an agenda specifically addressing ESOL. Practitioners are part of these 
two collaborations’ efforts but the voice of practitioners is combined with input 
from administrators, policymakers and researchers. This document, however, 
provides an opportunity to hear the direct voices of practitioners. 

The PDRN is a research and development effort working to link adult basic 
education practitioners and NCSALL researchers. The goal of the PDRN is to 
create and support systematic partnerships between practitioners and researchers 
in the field of adult basic education. The PDRN intends to: 

• help practitioners participate in NCSALL's work through involvement in 
NCSALL research studies, 

• disseminate information about NCSALL's studies and research results to 
practitioners, 
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• connect practitioner researchers and NCSALL researchers studying similar 
issues, 

• encourage practitioner research related to NCSALL research, and 

• provide training and financial support to practitioners to conduct inquiry 
projects related to NCSALL research. 

At the heart of the PDRN are the Practitioner Leaders. Practitioner Leaders, who 
are adult basic education teachers in each participating state, serve as liaisons 
between practitioners and NCSALL. The Practitioner Leader works with support 
from state Department of Education and State Literacy Resource Center 
representatives to facilitate the sharing of information between practitioners and 
researchers. Currently, nine Practitioner Leaders work in two regions of the 
United States, the Southeast and the Northeast. Practitioner Leaders are located in 
Georgia, Kentucky, Maine, Massachusetts, New Hampshire, Rhode Island, 
Tennessee, Vermont and Virginia. The focus groups were conducted by these 
Practitioner Leaders. 

The PDRN focus groups gathered data on three related questions: 

1 . What issues concern practitioners in adult basic education? 

2. How do practitioners see these concerns being addressed by current and 
future research? 

3. What do practitioners see as the role of the PDRN in their states? 

In this report we first review the process used in these focus groups and the ways 
the results have been analyzed. Then we examine the findings about areas of 
concern to the participants. Next we examine the findings that address the 
participants’ views on research and how research can best meet their needs. 

Finally we offer our conclusions and recommendations. 

II. Methodology 

A. Focus group format 

NCSALL PDRN staff (a national coordinator at World Education, and regional 
coordinators at World Education in the northeast and The University of Tennessee 
in the southeast) developed the protocol for the three hour focus groups. 
Practitioner Leaders received a five-page protocol and training guide from PDRN 
staff. Practitioner Leaders were asked to identify five to eight practitioners 
representative of their state’s literacy program staff, from urban and rural, ESOL, 
volunteer programs, etc. These practitioners were invited to take part in the focus 

2 
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group and were paid $75 ■ $100 for their participation. The Practitioner Leader 
identified another person to help with the facilitation and documentation of the 
focus group. The sessions were audiotaped and the facilitators took written notes. 
The focus groups were conducted in June and July, 1997. The number of 
participants in each group varied from three to ten; most were teachers, but 
program directors, counselors and other program staff were also represented. (See 
Appendix A for a description of each group.) Each Practitioner Leader wrote a 
report on the focus group. The PDRN staff at World Education collected the 
audiotapes and the reports. 

The process for the focus groups included the following steps: 

1 . An introduction by the facilitator to the purpose of the focus group. 

2. A discussion of how a focus group is conducted and basic ground rules for 
participation in the focus group. 

3. An “affinity diagram” process (see Appendix A), in which individuals 
identified questions or problems they were facing in their work. Then, 
working in small groups, the problems or questions that were similar (had 
“affinity”) were grouped and labeled by a phrase that categorized that group. 

4. An overview, by the facilitator, of the research currently being conducted or 
planned by NCSALL. 

5 . A group discussion of how such research might address practitioners’ 
questions and concerns, and what other research is needed in the future. 

6. A solicitation of suggestions of how research results might best reach 
practitioners in order to help them with their questions or problem. 

7. Development of two to three critical research questions on areas of concern 
not currently being addressed by NCSALL. 

8. A discussion of the role of the PDRN in their state. 

(The complete protocol is found in Appendix B.) 

Without exception the Practitioner Leaders were positive about the experience of 
facilitating the groups. 

B. Analysis 

There have been three stages of analysis of the data from the focus groups. First, 
the Practitioner Leaders used their notes from the focus groups to develop written 
reports that included sections on: 

• the categories from the affinity diagram, 
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• ways in which participants felt NCSALL research could help them address 
their problems, 

• ways in which NCSALL research could best reach them, 

• specific questions to add to the ABE research agenda, 

• ideas for the PDRN, 

• comments on the focus group. 

While all Practitioner Leaders followed this report format, the amount of detail in 
each section varied considerably from report to report. Some reports summarized 
the information in each section while others included each comment. A few focus 
groups did not cover every activity outlined in the guide. 

In July 1 997 the Practitioner Leaders met in Boston with PDRN staff and 
conducted a second analysis of the results from the nine focus group reports. In 
small groups, the Practitioner Leaders took the information from a particular 
activity in each focus group and analyzed it, looking for categories and trends. 

The similarities and differences among and between focus groups were then 
discussed in a large group. The categories they found provided a starting point for 
the next phase of the analysis. 

For the third stage of analysis reported here we have combined the data from the 
nine focus group reports. We used qualitative analysis to identify categories in 
the data and grouped these categories into two broad areas: (1) the problems and 
concerns of the participants that might be topics of research and (2) comments on 
how research findings can be most useful to the participants. We have identified 
subcategories in most of the categories in each broad area. 

The broad area of problems and concerns includes data from activities in the focus 
groups where practitioners do affinity diagrams and pose questions to be added to 
the research agenda. In the focus groups each participant had time to think of 
questions or problems that they face in their work— broad areas of concern more 
than specific issues. Then as a group, the participants combined the questions or 
issues that had some “affinity” and labeled each with a phrase that categorized the 
problem represented by that group. In their written reports the Practitioner 
Leaders reported these categories and the questions and problems in each. 

The Practitioner Leaders’ reports contained a total of sixty-three affinity 
categories, with number per focus group varying from four to thirteen. There was 
some overlap in these categories by title across the nine reports; however, when 
the items that made up the categories were examined, this overlap did not always 
hold up. In an effort to get a sense of the entire data set, our analysis aggregated 




4 



NCSALL Reports #4 



July 1998 



everything reported in the affinity diagram section of the reports. We identified 
eight categories from these data and coded the combined data set (affinity 
categories and comments from all nine reports). 

In the findings we report the numbers of comments in the various categories, in 
order to give a sense of how often particular issues arose, and we give examples 
of the concerns raised. These numbers should be used with some caution due to 
the variety in the original reports. The total number of affinity diagram items 
reported per state ranged from four for Vermont to sixty-two for New Hampshire. 
The numbers for each category are summarized in Appendix C. 

During the focus groups, participants identified research questions they felt might 
not be addressed by the four broad questions of NCSALL’s current research 
projects: 

1 . How can the motivation of individual adult learners be sustained and 
enhanced? 

2. How can classroom practice be improved? 

3. How can staff development more effectively serve adult learning and literacy 
programs? 

4. What impact does participation in adult learning and literacy programs have 
on an adult’s life and how can this impact be effectively assessed? 

The groups identified critical u gap" areas of concern from their affinity diagram 
process which they thought were not covered by current NCSALL research 
projects and then developed research questions for future research in that area. We 
have sorted these questions using the same categories as with the affinity diagram 
data. Because some research questions parallel comments drawn from the affinity 
diagram discussion, some overlap exists. All are included in Appendix D. 

The second broad area of analysis focuses on the usefulness of research. In 
facilitating the focus groups, after the affinity diagram process, the Practitioner 
Leaders asked their groups to discuss how NCSALL’s research might help them 
address their concerns; in other words, how research could be most valuable. Then 
the discussion focused on how results of this research could reach them in their 
classrooms so that it would really help them address their problems. The first 
question was intended to focus on the way the planned research should be 
conducted to be most useful and the second question focused on dissemination of 
the research. The Practitioner Leaders were asked to report their results in the 
same way. However the reports (and probably the discussions) mixed these two 
areas of discussion. We have done the same in this report, combining the 
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responses to, “Ways in which participants felt NCSALL research could help them 
address their problems,” with, “Ways in which participants felt that NCSALL 
research could reach them in order to help them.” 

We have grouped the responses in three categories. Twenty-four of the responses 
dealt with topics for research; ten were comments on the research process; and 
ninety-two were suggestions for how to disseminate the research. Each of these 
categories was sorted into subcategories as reported in the findings. 

III. Findings 

A. Areas of concern and research questions 

After we combined the affinity group data from the reports, we identified phrases 
and research questions reflecting particular issues, and from these 241 fragments 
we identified eight categories. 

1 . Recruitment, retention, motivation (58) 

2. Program and policy issues (41) 

3. Curriculum and instruction (28) 

4. Assessment and standards (27) 

5. “Special” students (26) 

6. Staff issues and professional development (24) 

7. Impact on students’ lives outside the classroom (21) 

8. “Integrated” classrooms (16) 

Within each of these categories additional subcategories were identified. In these 
subcategories there is some overlap, and some statements were assigned to more 
than one subcategory. 

This process is somewhat arbitrary and certainly different categories could be 
obtained. (For example, a separate category was not reported for references made 
to ESL or ESOL, as all ESOL-related items could be placed in existing 
categories.) But this preliminary report should give a sense of the concerns of 
these nine groups of practitioners. The following sections review the categories 
drawn from the affinity diagram process data and corresponding research 
questions and give examples, in the form of direct quotes, from the nine reports 
(in italics) and the number of items (in parentheses) regarding the topic brought 
up during the focus group. 
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1. Recruitment, retention, motivation, participation 

The greatest number of comments and questions were about issues of student 
participation: how to get people into class, how to keep them there, how to 
motivate learning, and how to deal with barriers to learning. This was an issue in 
eight of the nine groups and had the largest number of comments in four states. 

a. Main concerns (58) 

Recruitment (6) 

• If we are to "market" our product, how do we need to offer it up to our 
target population, who are basically not-in-need? (GA) 

Retention (14) 

• Retention - What types of things work? Learning contracts? Bonding? 
Student Support Groups? (KY) 

• Why do some students stop coming to class after showing great interest in 
the beginning ? (TN) 

Motivation (22) 

• How to motivate students to participate and follow through with their goals 

(ME) 

• How to motivate ADD students - when there are time constraints? (NH) 
Barriers to participation (16) 

• Obstacles to students learning, including experiences from the native 
country. (MA) 

• How come some students' problems outside of school have too big an 
influence on their ability to come to school? (NH) 

b. Research questions (8) 

• How can ABE students be motivated to learn new things with all the 
distractions of their lives? (NH) 

• What effect does the lack of sufficient facilities have on recruitment and 
retention? (GA) 

2. Program and policy issues 

This category contains concerns that relate to issues that tend to be decided on a 
program or policy level or with system operations at a level beyond the 
classroom. With forty-one references, this is the second largest category. 

a. Main concerns (41) 

Time and money (14) 

• What ’s a good way to obtain adequate, ongoing (not yearly) funding for 
programs? (V A) 

• Why are adult ed salaries generally so low? (VT) 
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Program structure and curriculum (11) 

• Are programs realistically preparing students for the 21st century? (Rl) 

• Is an open-ended enrollment policy the most effective way to enroll 
students and structure programs? (NH) 

Systems, administration (6) 

• Paper work concerns: What should be included in intake? Attendance 
records, separation sheets; is all this paperwork necessary? Can record 
keeping be made easier? (ME) 

Relationships outside ABE/ESOL (10) 

• How can Adult Ed form coalitions with other agencies to maximize 
resources? (VA) 

b. Research questions (9) 

• What are the characteristics of a model adult education program, and what 
are the costs? (ME) 

• Compare/contrast adult education service systems. Points of comparison 
might include delivery models, funding distribution streams, quality 
indicators, credentialing. (VT) 

3. Curriculum and instruction 

This category includes concerns about how and what to teach. These twenty-eight 
issues were subcategorized as follows: 

a. Main concerns (28) 

Uses of technology (4) 

• How do we plan for and include technology in curriculum? (RI) 

Instructional strategies (11) 

• How can I help students to move on from a plateau? (MA) 

• How to administer group-oriented lessons when members of the group 
attend inconsistently? (NH) 

Thinking skills (3) 

• How do we develop thinking skills? (KY) 

Math and science (3) 

• How can we strengthen the teaching of mathematical concepts? (NH) 

Reading and language (7) 

• What are good techniques to encourage low level reading students to read outside 
the .classroom? (MA) 

b. Research questions (2) 

• What are the (most) appropriate uses — if any — of technology in adult ed? (VT) 

• What are the most effective teaching and assessment methods for each of the 
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various described multiple intelligences (assumes building off the present multiple 
intelligences study)? (VT) 

4. Assessment and standards 

The participants in these focus groups thought about assessment in a variety of ways. 
Some asked for assessment in order to better place students. Others were interested in 
measuring learning gains or the outcomes of participation in their programs. We have 
included concerns about standards in this category because standards are part of the 
broader system of performance assessment. 

a. Main concerns (27) 

Placement (5) 

• Is there a good short tool for placement purposes which is valid and non- 
threatening? (VA) 

Program effectiveness (2) 

• How can we evaluate the effectiveness of different types of curriculum? (MA) 
Measuring learning gains (11) 

• Lack of definition of "progress of learning" for our learners (ME) 

• Evaluating - in ESL - simple standard tests - are they available so that the 
students can see where they begin and where they are going? (NH) 

Impacts and outcomes (6) 

• How do we know how students use what we teach them? (MA) 

Standards (3) 

• Incorporating "standards" into adult ed programs: Whose standards? What 
standards? How? Why? (ME) 

b. Research questions (7) 

• How do we as educators know what our adult learners are learning, when they 
are using it and what form of assessment should be used? (MA) 

• What level of learning or skill does a student need in order to use portfolio 
assessment as a means of assessment? How does a student self evaluate? (ME) 

5. “Special” students 

We include here concerns about students with learning disabilities or limited progress, 
about how to help people with emotional difficulties, and concerns about teens and the 
elderly. 

a. Main concerns (26) 

Teenagers (5) 

• Should there be admissions requirements for teens? Counseling? (NH) 

Elderly (2) 

• How do we address the difficulties faced by elderly students? (NH) 
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Mental health concerns (3) 

• How do we help students who ’ve suffered severe trauma - loss, war, re- 
settlement? (VA) 

Students with limited progress and learning disabilities (16) 

• How to assist the student with his/her learning disability (ME) 

• How can we better serve students who show no learning progression from year to 
year? (TN) 

b. Research questions (10) 

• How do we know whether there is a learning disability or a language barrier that 
is hampering an ESOL student? (MA) 

• What are the best approaches to help adults with learning disabilities? (VA) 

6. Staff issues and professional development 

This category includes concerns about the work conditions of practitioners and concerns 
about staff development in general. This category also includes references to assessing 
the effectiveness of teachers and to staff development around a particular issue, e.g. 
learning disabilities or reading instruction. 

a. Main concerns (24) 

Assessing effectiveness as a teacher (5) 

• What is the professional and personal profile of an effective teacher? (GA) 
Credentialing (5) 

• Teacher training and validation: impact of certification for adult ed teachers? 
(VT) 

Method and content of staff development (11) 

• How do we develop effective staff development programs and methods? (KY) 
Professional development system issues (3) 

• A lack of a system for teacher staff development to lead to teacher certification 

(ME) 

b. Research questions (4) 

• Does the education level of instructor effect student achievement? Educational 
level vs. teacher certificate (KY) 

• What is an effective practitioner? How can I assess my effectiveness? Are full-time 
practitioners more effective than part-time? If so, how ? If they are more 
effective, then how can we move toward full-time employment? (VA) 

7. Impact on students’ lives outside the classroom 

This category includes questions and issues that practitioners raised about the impacts of 
their work on students’ lives. These are concerns about how participation in adult literacy 
programs can make a difference in the lives of adult learners outside the class. 

10 
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a. Main concerns (21) 

Concerns about the value of adult basic education in people’s lives (5) 

• Are we really preparing people or are we giving them more superficial treatment? 

(GA) 

Concerns about how to best prepare people for employment (4) 

• Transfer of knowledge from classroom to life - How do we teach higher level 
thinking skills? (KY) 

Skills people need in their lives (12) 

• How do we teach decision-making? Social communication? (VA) 

• How do we as instructors assist students with coping skills? (GA) 

b. Research questions (3) 

• Social issues and attitudes regarding the value of education: What are businesses 
expecting from our GED graduates? What would make them more eligible for the 
job? (GA) 

• What can we do in our classrooms that will help our students break the low 
achievement cycle in their families? (NH) 

8. “Integrated” classrooms 

This category is similar to the previous one, but we decided to separate the two because 

the integrated classroom referred to issues of dealing with a variety of learners in one 

class, rather than problems with particular students’ learning. 

a. Main concerns (16) 

Multi-age classes (3) 

• Applied to ages and/or levels? Does the process of having a wide variation of 
student ages and levels of competency in the same classroom have a bearing on 
the learning process? (GA) 

Multi-level classes (10) 

• What are some effective practices to manage a multi-level class including 
curriculum and assessment? (MA) 

Multi-cultural classes (3) 

• How do we address multi-cultural and economic backgrounds - respecting 
differences? (NH) 

b. Research questions (4) 

• Explore the variety of effective technology and curriculum for adult non-readers 
and determine applicability to low-level readers , learning disabilities, mental 
retardation, multilingual, various cultural backgrounds and regional differences. 

(GA) 

• What is the best way to teach reading in a multi-level classroom? (NH) 

11 
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Welfare reform 

There were also three comments that referred directly to welfare reform. This is an issue 
that is having an increasing impact on adult education programs. These comments were: 

• How to deal with welfare reform and its discouragement of education (MA) 

• How can we provide short , intensive instruction for Welfare Recipients (due to 
welfare reform) when they typically have the biggest educational deficits and 
learning problems? (NH) 

• How will welfare reform impact enrollment in adult ed classes? (Rl) 

B. Usefulness of research 

Part of the focus group discussions centered on how results of research could reach 
practitioners in their classrooms in ways that would really help them address their 
problems, as well as on ways the planned research should be conducted in order to be 
most useful to practitioners. 

We have grouped the responses in three categories: topics for research; improving the 
research process; and suggestions for research dissemination. 

1. Topics for research 

The comments on topics for research were categorized using the same categories as in the 
areas of concern from the affinity diagram. These are all responses to the question about 
how NCSALL research could help them. The number of comments from each category 
are listed after the topic. 

• Impact on students’ lives outside the classroom (5) 

• Program and policy issues (5) 

• Recruitment, retention, motivation (5) 

• Assessment and standards (4) 

• Curriculum and instruction (2) 

• Staff issues and professional development (2) 

• “Integrated” classrooms (1) 

• “Special” students (0) 

For the most part these suggestions either referred to specific NCSALL studies or were 
comments similar to those in the previous sections. However the suggestions from 
Georgia and Vermont on program or policy concerns give a perspective on what the 
practitioners in these focus groups hope for from research and research establishments: 
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The research would be valuable if it... 

• lends credibility to our program. 

• leads to increased funding/resources. (VT) 

NCSALL should... 

» become a baseline information source for assistance in data reporting for 
grant application. 

• provide guidelines for policy writing. 

• generate ideas for improved programs. (GA) 



2. Improving the research process 

We found ten comments on the research process in the responses to how the research 
could be most helpful. These focused on how to conduct research so as to make it “real.” 
These include comments on site and focus: 

• Includes rural areas in research as well as urban areas. (ME) 

• Includes real students in real classrooms with real challenges. (MA) 

• Has a holistic focus: whole person, families. (VT) 

There were also comments on who should be doing the research, comments that were 
echoed in suggestions about how to best reach the classroom with dissemination. 

• [If research ] had been done by people who actually work in classroom. (NH) 

• Could practitioners be paid to field-test research findings to check on their validity? 
(VA) 

3. Suggestions for dissemination of research 

These focus groups were as productive in offering suggestions for dissemination of 
research as they were in expressing their problems and issues of concern. We identified 
92 suggestions from this discussion on ways to increase the likelihood that NCSALL 
research results will make a difference to the field of practice in adult literacy education. 
Fifty-three of these were suggestions about methods of dissemination. The following are 
the dissemination methods suggested. 

• print publications (1 1) 

• workshop presentations (10) 

• television and video (6) 

• presentations at conferences (5) 

• on-line, listserv, Internet (5) 
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• interactions with researchers (5) 

• “live” presentations (4) 

• peer discussion (3) 

• networking (2) 

• audio tapes (1) 

• visual aids (1) 

There were fifteen comments about the preferred language and format for research 
products. Some of these were: 

• Is user friendly. (ME) (VT) 

• Needs to be more detailed. (MA) 

• Is reported in clear, every-day language. (VA) 

• Take research and rewrite for a general audience. (KY) 

• Something short and concise. (TN) 

• Written in easily understood, interesting layperson's terms. (NH) 

• Is not steeped in statistical data. (VT) 

The other comments on dissemination had to do with ways to make research results 
useful to teachers. These twenty-four comments focused on practicality and on concrete 
applications in the classroom. Some examples: 

• [It would be helpful if it] had recommendations or suggestions on what to do with it, 
if it uses resources that are readily available and inexpensive and practical. (ME) 

• Research needs to have clear practical implications, not just statistics. Hard to 
analyze quantifiable research and do mentoring. I have questions all the time, I want 
positive steps to take. (MA) 

• Demonstrates effective, concrete teaching models and methods. (VT) 

• Need practical applications for research. (KY) 

There were also suggestions for teacher research in developing and piloting applications 
and for teacher research as a way to ensure that research meets the needs of practitioners. 
For instance: 



Pilot new methods in model learning centers (KY) 

Opportunities and activities to help teachers test it out - bring theory to the classroom 
in a concrete way. (MA) 

Practitioners in Virginia would like to be involved in the whole research process, 
developing questions, collecting data, trying out new techniques. (VA) 




14 



21 



NCSALL Reports #4 



July 1998 



C. The role of the PDRN 

The final activity in the focus groups was a discussion about the role of the PDRN in each 
state. The comments were suggestions directed to the particular state in which each focus 
group was held, rather than the to the PDRN as a national network. It is worth noting that 
of the fifty-three various comments and suggestions, nineteen were about teachers being 
involved in research. Practitioners in these groups thought that the role of the PDRN 
included: 

• Informing practitioners of possible grants available for practitioner research. (ME) 

• Have practitioner leader meet with groups across the state to help set up and 
facilitate research groups. (KY) 

• Create ways and opportunities for teachers to share their own research, set goals for 
programs, give periodic feedback on research in progress, help researchers to stay 
close to the practical realities of adult literacy. (MA) 

IV. Conclusions 

The field of adult basic education is diverse: teachers serve non-readers; prepare students 
for the GED; provide ESOL instruction, native language literacy and citizenship classes; 
and teach in hospitals and factories. Teachers are part-time and full-time, experienced 
and new, work in urban and rural communities, and teach at government-funded and 
privately-funded programs with varying degrees of support and channels for resource 
sharing. Teachers, administrators, coordinators, counselors, and program directors 
participated in the focus groups. A handful plays more than one of these roles in their 
programs. This allowed for a variety of perspectives and resulted in a wide range of 
concerns. 

Eight categories of common concern emerged from the focus group reports: impact on 
students’ lives outside of the classroom; curriculum and instruction; “special” students; 
“integrated” classrooms; staff issues and professional development; recruitment, 
retention, motivation, and participation; program and policy issues; and assessment and 
standards. 

Of the eight categories, staff issues and professional development proved to be a constant 
concern, especially in questions regarding how staff development should be structured. 
Concerns around the issues of recruitment, retention, motivation and participation were 
felt most often, with emphasis placed on strategies for keeping students in class, 
motivation techniques and identifying obstacles that may possibly inhibit students’ 
learning. There was discussion around issues of program funding, program structure, 
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curriculum, learner assessment, learning disabilities, the multi-level classroom and 
teaching life skills. 

Practitioners identified topics they would like research to address. Among the most 
common were the following: learning disabilities, model programs and program support 
systems, and student and teacher assessment. In thinking about future research, 
practitioners felt they would benefit most from research done on the following topics: 
impact on students’ lives outside the classroom; recruitment, retention, motivation, and 
participation; and program and policy issues. 

Many suggestions were made in the focus groups about improving the research process 
and disseminating research to practitioners in a useful way. Practitioners believe that 
research can be improved by allowing teachers and students to take part in a study, 
conduct practitioner research on a similar topic, or pilot the results. There is a strong 
feeling from practitioners that this would keep research in touch with the realities of the 
classroom. Recommendations for disseminating research were many. Practitioners 
agreed that the language of research reports intended to reach programs needs to be clear, 
concise and written for a general audience. They would find research results especially 
useful if concrete classroom applications accompanied research results. Among the many 
ideas brainstormed, the following methods of disseminating research results were thought 
to be the most productive: publications, workshop and conference presentations by 
researchers or research assistants, television/video, the Internet, and peer discussion. 

Discussion about the role of the PDRN focused primarily on individual states’ needs, 
although the theme of teacher research appeared often. Practitioners hope Practitioner 
Leaders will help teachers use existing research, find funding for practitioner research, 
and create opportunities for teachers to share their own research. 

The analysis process used in this report is not intended to explore all the possible 
interpretations of this data but to give the field, NCSALL and others involved in setting 
research agendas a sense of the concerns expressed by participants in these focus groups. 
Practitioners in adult basic education are looking for answers and stepping stones to 
improve practice. This report shows practitioners’ enthusiasm for being involved in 
research, the strength of their demand for research and their desire for collaboration 
between the field of adult basic education and academia. 
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Appendix A 

Protocol for NCSALL focus groups: 3 Vi hours 

1 . Introductions, purpose of th is focus group ( 1 0 minutes) 

• Welcome everyone to the focus group. Ask each person to introduce 

themselves, the program they’re from, their role (teacher, counselor, etc.). 

Explain to the group the purpose of the group, using the following points 
written on newsprint: 



NCSALL Focus Group: Objectives 

• Hear about your issues and concerns as practitioners in adult basic 
education 

• Help you learn about NCSALL and its research activities 

• Ask you to think of critical areas for research 

• Get your thoughts about NCSALL ’s efforts to include practitioners in 

research and dissemination (Practitioner Dissemination and Research 
Network) 



Ask if anyone has questions or comments. Explain that you know that 
they probably don’t have much idea yet about what NCSALL is but that 
you will be providing an overview during the time together. Deal with 
immediate logistics (where bathrooms are, there will be a break, you’ll 
answer questions about stipends, etc. at the end...) 

2. What is a focus group? Ground rules (1 5 minutes) 

Ask if anyone in the group has ever been part of a “focus group” before. If 
so, ask them to say very briefly what that experience was like. Were they 
nervous? etc. Explain the following points about focus groups: 

• The purpose of a focus group is to get information and ideas from you. 
Your experience is valid. NCSALL is interested in hearing what you have 
to say, and there is no one “right” answer. 

• -The goal of the focus group is to hear equally from everyone. It won’t just 

be one large discussion for 3 hours; there will be a variety of activities, and 
sometimes you will work in small groups, sometimes there will be an open 
discussion, and sometimes everyone will be asked individually to say what 
they think. We encourage people to monitor themselves so that everyone 

17 




24 



gets a chance to talk in open discussions. 



• The information from this focus group will be used by NCSALL staff to 
help improve their activities. Some of the information will be 
incorporated into a larger draft agenda for research that is being developed 
over the course of the next year, and which will help to drive research in 
adult basic education over the next 10-15 years. Most of NCSALL’s 
research money is already allocated to particular research projects (which 
you’ll hear about) so the purpose of this focus group is not to define 
NCSALL’s research during the life of its grant but to give input to a long- 
term agenda for adult basic education research which all funders (DOE, 
NIFL, etc.) can buy into. 

• All information from this focus group will be confidential; that is, no one’s 
name or the name of their program will ever be written in the notes. This 
is one of nine focus groups going on in four states in the south and five in 
New England, and the information from all of the focus groups like this 
will be compiled and presented as one big report. 

• We’d like permission to tape this focus group, just as a back-up record in 
case we miss anything in our notes, but the primary form of record will be 
the notes that the facilitators keep. There are two facilitators; one will be 
leading the activity or discussion and the other will be taking notes. The 
results of the focus group will be typed up from their notes, but the tape 
could provide a valuable backup in case the notes are unclear, etc. Does 
anyone have any objection to being taped? If so, we don’t need to do it. 

• Here are some basic ground rules for focus groups: Everyone tries to 
contribute equally. We want to hear from everyone. Try not to interrupt 
others. Listen carefully to what others are saying. Ask questions about the 
process if at any time you are confused about what we’re doing or why... 
OTHERS? 

3. Affinity Diagram: Issues and concerns (40 minutes) 

Explain to the group: This first activity is designed to get from you what 
your main concerns are in your work. Everyone should take a small stack 
of post-its (hand out) and on each post-it, write one question or problem 
that you are facing in your work. Here we are asking for broad concerns 
(things like “How do I get student Tom to come to class regularly?”, can 
be stated as “What makes some students come to class regularly, others 
irregularly, and what can I do about it?”) Take about 5 minutes silently, by 
yourself, to think of 3-4 questions or problems, and write them on the 
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post-its, one per post-it. Write legibly and as largely as possible on the 
post-its, as other people will be reading them. 

Then, come together in a group of up to five people (if there are five 
people in total, make one group; if there are eight in the focus group, make 
two groups of four, etc.) Everyone stick their post-its on a big sheet of 
newsprint in no particularly order. Take a few moments to read all of the 
post-its. Then, as you talk with your group members, move “like” post-its 
together so that similar problems or questions are placed next to each other 
in groups. Continue until every post-it is part of a small group of post-its 
that has an “affinity”. Then, take a marker and label each group with a 2-4 
word paraphrase that categorizes what the overall problem or question is 
for all those post-its; e.g., “learner persistence” or “assessment of learning 
outcomes”. 

As a whole group, have a brief discussion of what you found out. If there 
are two groups, post both newsprints and ask people, during a short break, 
to read the other group’s notes. Discussion should focus on where the 
similarity is in problems/questions. Keep newsprint posted for remainder 
of focus group. 

Overview of NCSALL and its research (30 minutes) 

Explain that you will give a brief overview of NCSALL, based on all the 
knowledge you have, about what NCSALL’s goals and activities are. Use 
the overhead or handouts attached to guide you through this discussion. 
Explain that people can ask questions but because you are not a member of 
the NCSALL staff you might not always have the answer but you will 
convey the questions you don’t know back to the NCSALL staff. Brief 
discussion 

Analysis of if and how NCSALL’s research is addressing vour concerns 
(30 minutes) 



Ask participants to look at the areas of NCSALL’s research (learner 
motivation, classroom practice, staff development, assessment), and then 
to look at their categories of questions and tell you whether there is an 
overlap. What areas of concern are related to or are being addressed by 
NCSALL’s research? Put a circle or a green star around these areas. 



Lead a discussion with the following questions: 

A. Ask participants to discuss, in the whole group, how NCS ALL’s 
research may help them address their questions or concerns; e.g., 
ask them to finish this sentence, “This research would be valuable 
if it.. .(focused on this aspect of the problem, included a particular 
population of students, etc...)” 

B. Ask participants to suggest ways in which the results of this 
research could reach them in their classrooms so that it really 
would help address their problems; e.g., fill out this stem “This 
research would really help me with my question or problem if 
it.. ..(came to me in a particular format...)” 

Next, ask participants to think about where there is a gap between their 
areas of concern and NCSALL’s research. What are the major categories 
of problems or questions that you have identified that aren’t being 
addressed by NCSALL? Put a circle or red star around the areas of 
concern that aren’t being addressed under NCSALL’s research. 

6. Developing 2-3 critical research questions (based on your concerns) for the 
long-term research agenda in ABE (30 minutes) 

For the areas that are gaps between their problems and NCSALL research, 
explain that these areas could be part of the long-term research agenda for 
adult basic education, but they need to be phrased as specific questions for 
inclusion in the draft agenda. Ask participants to get into pairs, and have 
each pair take one of the areas that has been circled or red starred and try 
to come up with a specific question. Perhaps it is one of the questions on 
the post-it note, or perhaps it is a broader question that includes all of the 
specific post-it notes under that category. Try to arrange it so that most of 
the “gap” categories are covered by one or another pair; i.e., if there were 5 
categories of problems/questions on the affinity diagram newsprint that 
were identified as not being addressed by NCSALL’s research, try to have 
each pair take one of those categories to work on. If possible, it should be 
a category that one or both of the partners is interested in. 

Give the pairs about 15 minutes to work on their question; they don’t need 
to write it on newsprint, but it does need to be written out. Go around and 
ask each pair to read off their question, which the facilitator then writes up 
on newsprint. The wording doesn’t have to be perfect, it just needs to 
convey the basic idea of what future research should focus on in that area. 
Quick discussion 
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Overview and discussion of the PDRN (45 minutes) 



Explain the idea of the PDRN, using the concept paper attached. Answer 
questions about the idea, then lead a brief discussion asking people to talk 
about what they think the PDRN could do to link practitioners with 
NCSALL. Take notes. 

Thank people for their participation . Explain again what will happen with 
all the information (typed up in notes, compiled with other focus group 
notes, included in draft research agenda, sent to NCSALL staff to review 
and discuss how to take these ideas into account for their research and for 
the PDRN). 



Format for writing up focus group notes: 

Categories from Affinity Diagram and questions underneath (from Activity 3, 
from the newsprint/post-its): Please star once those categories that participants 
felt NCSALL was addressing in their research, and star twice those categories that 
participants felt NCSALL was not addressing in their research. 

1 . 

2 . 

3. 

4. 



5. 



Wavs in which participants felt NCSALL research could help them address 
their problems (from Activity 5A, from your notes): 



Ways in which participants felt that NCSALL research could reach them in 
order to help them (from Activity 5B, from your notes): 



Specific questions that should be added to the draft ABE research agenda 

(Activity 6, from newsprint): 



Ideas for the PDRN (from Activity 7, from your notes): 



Your comments and impressions from the focus group : What did you think 
went well? What didn’t work? What struck you most about their comments? 
What would you stress from your own experience? Any other comments? 



Appendix B 

Participants in focus groups according to state 



State 


Number of 
Participants 


Female 


Male 


Teacher* 


Administrator/ 
Coordinator/ 
Counselor/ 
Program Director* 


Georgia 


8 


5 


3 


8 


0 


Kentucky 


7 


5 


2 


3 


4 


Maine 


5 


5 


0 


5 


2 


Massachusetts 


7 


6 


1 


7 


0 


New Hampshire 


10 


9 


1 


8 


2 


Rhode Island 


8 


5 


3 


7 


2 


Tennessee 


3 


2 


1 


2 


1 


Vermont 


7 


5 


2 


6 


2 


Virginia 


8 


8 


0 


8 


1 


TOTALS 


63 


50 


13 


54 


14 



* numbers are duplicated for participants performing two roles 
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Appendix C 

Number of affinity diagram items by category and state 
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Category 


1 . Recruitment, retention, 
motivation, participation 


2. Program and policy issues 
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4. Assessment and standards 


5. “Special” students 


6. Staff issues and professional 
development 


7. Impact on students’ lives 
outside the classroom 


8. “Integrated” classrooms 


Totals 
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Appendix D 

Questions for the research agenda 



"Special” students (10) 

• How do we know whether there is a learning disability or a language barrier 
that is hampering an ESOL student? (MA) 

• How do you develop and document student education plans for special 
populations, i. e. ESL, aging population, and learning disabled? (KY) 

• How to diagnose a learning disability in an adult and strategies to instruct the 
adult? (ME) 

• What teaching methods are best to work with adult students who have 
experienced trauma, who have little sense of "self ’? (MA) 

• How can research direct teachers of students with developmental, linguistic 
and/or mental health issues to succeed in adult classes? (NH) 

• How can researchers address the needs of teens vs. adults in adult classes? 
(NH) 

• How can we best provide educational and social services to the non- 
traditional learner: assess the needs, establish individual education programs 
and make referrals to programs which would best serve them? ( RI) 

• Expand — or ensure expansion of — learning disability research to ADULTS. 
(VT) 

• What are the best approaches to help adults with learning disabilities? (VA) 

Program and policy issues (9) 



• How are adult learners impacted by successful collaboration of adult ed and 
social service agencies? (ME) 

• What are the characteristics of a model adult education program and what 
are the costs? (ME) 



What are good models for building strong community support for adult 
education? (MA) 
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• Given that some students may be incapable of passing the GED exam, how 
can ABE Programs serve these students ’ needs (by providing other options 
and maintaining social support)? (NH) 

• Is funding available for increasing classroom time and obtaining classroom 
supplies? How does increasing class length and/or increasing the school year 
affect student learning? (NH) 

• Can we do a comparative study of learners from across the country - acquire 
and disseminate information on characteristics and similarities of ABE 
learners — their needs, goals, ambitions — to form a national level portrait of 
the adult learner? (RI) 

• How can we develop a better intake system - can we research what interview 
methods are successful and what an interviewer should know about a learner 
to ensure that the learner will be appropriately placed? (RI) 

• Compare/contrast adult education service systems. Points of comparison 
might include delivery models, funding distribution streams, quality 
indicators, credentialing. (VT) 

• What approach will promote a realistic view and understanding of the value 
of adult ed in the community? (VA) 

Recruitment, retention, motivation, participation (8) 

• How can ABE students be motivated to learn new things with all the 
distractions of their lives? (NH) 

• What help is available to motivate ADD students to learn in the classroom? 
(NH) 

• How do we encourage/validate the long term student (basic ed) who won't be 
taking the GED for a long time or ever? (NH) 

• How can we develop an effective recruitment strategy to enroll basic 
learners? (NH) 

• Can creative solutions be found to remove roadblocks that are preventing 
attendance? (NH) 

• How do we assess the retention ability of students and how do we adjust 
teaching strategies to improve retention ability? ( KY) 
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• Examine retention parameters regarding male and female adult students 
subdivided into two basic age brackets: 16 to 19 years and above 19. (GA) 

• What effect does the lack of sufficient facilities have on recruitment and 
retention? (GA) 

Assessment and standards (7) 

• How can we accurately assess the reading levels of our students? (NH) 

• Could a simple standard test be developed so ESOL students could evaluate 
themselves? (NH) 

• How do we as educators know what our adult learners are learning, when 
they are using it and what form of assessment should be used? (MA) 

• What are some learner-supportive ways to measure progress towards 
individual learner’s goals/needs in a multi-level class? (MA) 

• What are all the possible standards models available and how are they 
incorporated into adult education standards that will inform classroom 
instruction? (ME) 

• What level of learning or skill does a student need in order to use portfolio 
assessment as a means of assessment? How does a student self evaluate? 

(ME) 

• What are the most effective methods of assessment with the adult learner at 
varying levels? (ME) 

"Integrated” classrooms (4) 

• What is the impact on learning of the tension between different cultural and 
sub-cultural backgrounds and the dominant U.S. culture? Provide teachers 
with effective mechanisms and tools to bridge cultural gaps. (V A) 

• Explore the variety of effective technology and curriculum for adult non- 
readers and determine applicability to low-level readers, learning disabled, 
developmentally disabled, multilingual, various cultural backgrounds, and 
regional differences. (GA) 

• What is the best way to teach reading in a multi-level classroom? (NH) 
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• What are some effective practices to manage a multi-level class including 
planning, curriculum and ongoing assessments? (MA) 

Staff issues and professional development (4) 

• What constitutes effective adult literacy teachers and/or staff on some 
measurable components of attitude, commitment, and equity? (GA) 

• Does the education level of instructor affect student achievement? 

Educational level vs. teacher certificate (KY) 

• What are appropriate means of evaluation for all staff (i.e.. administrators, 
teachers, support staff, etc.) and what impact does this evaluation have on the 
program design? How does the evaluation maintain the level of 
professionalism ? (ME) 

• What is an effective practitioner? How can I assess my effectiveness? Are 
full-time practitioners more effective than part-time? If so, how ? If they are 
more effective, then how can we move toward full-time employment? (VA) 

Impact on students’ lives outside the classroom (3) 

• "Real life link”: Is this measurable? What can we do to help? (GA) 

• Social issues and attitudes regarding the value of education: What are 
businesses expecting from our GED graduates? What would make them more 
eligible for the job? (GA) 

• What can we do in our classrooms that will help our students break the low 
achievement cycle in their families? (NH) 

Curriculum and instruction (2) 

• What are the (most) appropriate uses — if any — of technology in adult 
education? (VT) 

• What are the most effective teaching and assessment methods for each of the 
various described multiple intelligences (assumes building off the present 
multiple intelligences study)? (VT) 
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Appendix E 

Categories of issues of concern to focus group practitioners 



Category 


From affinity 
diagram 


Item for research 
agenda 


From research 
discussion 


1 . Recruitment, retention, 
motivation, participation 


52 


8 


5 


2. Program and policy issues 


36 


9 


5 


3 . Curriculum and instruction 


29 


2 


2 


4. Assessment and standards 


27 


7 


4 


5. “Special” students 


26 


10 


0 


6. Staff issues and professional 
development 


21 


4 


2 


7. Impact of students’ lives 
outside of the classroom 


18 


3 


5 


8. “Integrated” classrooms 


16 


4 


1 
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MCTALL 



National Center for the Study of Adult Learning and Literacy 



The Mission of NCSALL 

The National Center for the Study of Adult Learning and Literacy (NCSALL) will pursue basic 
and applied research in the field of adult basic education, build partnerships between researchers and 
practitioners, disseminate research and best practices to practitioners, scholars and policymakers, and 
work with the field to develop a comprehensive research agenda. 

NCSALL is a collaborative effort between the Harvard Graduate School of Education and World 
Education. The Center for Literacy Sturdies at The University of Tennessee, Rutgers University, and 
Portland State University are NCSALL’s partners. NCSALL is funded by the U.S. Department of 
Education through its Office of Educational Research and Improvement (OERI) and OERI’s National 
Institute for Postsecondary Education, Libraries, and Lifelong Learning. 

NCSALL’s Research Projects 

The goal of NCSALL’s research is to provide information that is used to improve practice in 
programs that offer adult basic education, English to Speakers of Other Languages, and adult secondary 
education services. In pursuit of this goal, NCSALL has undertaken research projects in four areas: (1) 
learner motivation, (2) classroom practice and the teaching/leaming interaction, (3) staff development, 
and (4) assessment. 

Dissemination Initiative 



NCSALL’s dissemination initiative focuses on ensuring that the results of research reach 
practitioners, administrators, policymakers, and scholars of adult education. NCSALL publishes a 
quarterly magazine entitled Focus on Basics', an annual scholarly review of major issues, current 
research and best practices entitled Review of Adult Learning and Literacy ; and periodic research reports 
and articles entitled NCSALL Reports. In addition, NCSALL sponsors the Practitioner Dissemination 
and Research Network, designed to link practitioners and researchers and to help practitioners apply 
findings from research in their classrooms and programs. NCSALL also has a web site: 

http://hugse I .harvard.edu/~ncsall 

For more information about NCSALL, please contact: 

John Comings, Director 
NCSALL 

Harvard Graduate School of Education 
101 Nichols House, Appian Way 
Cambridge, MA 02138 
(617) 495-4843 
ncsall@hugse I .harvard.edu 
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NCVALL 



National Center lor the Study of Adult Learning and Literacy 



Reports 

Reports are NCSALL articles and research findings that can be purchased through World 
Education. The following reports are available: 

1 . Merrifield. J. (1998). Contested Ground: Performance and Accountability in Adult 

Education. 

2. Purcell-Gates. V., Degener, S., & Jacobson, E. (1998), Adult Literacy Program 
Practice: A Typology' Across Dimensions of Life-ContextualizedJDecontextualized and 

Dialogic/Monologic. 

3. Tyler. J.. Mumane. R., & Willett, J. (in press), The Economic Impact of the GED 
(tentative title). 

4. Bingman. M., Smith, C., & Stewart, K. (1998), Practitioners Speak: Contributing to a 
Research Agenda for Adult Basic Education. 

5. Rudd. R.. Zacharia, C., & Daube, K. (in press), Integrating Health and Literacy: Adult 
Educators Experiences. 

6. Beder, H. (in press). The Outcomes and Impacts of Adult Literacy Education in the 
United States. 



Focus on Basics 

Focus on Basics is the quarterly publication of NCSALL. The following issues are available for 
purchase through World Education: 

1. Vol. 1, Iss. A (Feb. 1997), Research. 

2. Vol. 1, Iss. B (May 1997), Reading. 

3. Vol. 1, Iss. C (Sept. 1997), Multilevel Classrooms. 

4. Vol. 1, Iss. D (Dec. 1997), Content-Based Instruction. 

5. Vol. 2, Iss. A (Mar. 1998), Learner Motivation. 

6. Vol. 2, Iss. B (June 1998), The GED. 

7. Vol. 2, Iss. C (Sept. 1998), Theory of Change. 

All of the above materials can be ordered by contacting the NCSALL Dissemination Office, 
World Education, 44 Farnsworth Street, Boston, MA 02210-121 1; telephone (617) 482-9485; 
e-mail NCSALL@WorldEd.org. 
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